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ABSTRACT 

Measuring the effects of computer-managed instruction 
(CMI) on the teaching of and student writing about literature 
involves more than having students write and then evaluating their 
performance. Measurement is made difficult by the fact that the 
computer technology used in instruction is in a state of flux. 
Variation of computer technology, whether in the form of the 
operating system, platform, or instructional software, is profitable 
for its designers. The use of computers in literature instruction can 
means (1) greater creativity, because instruction is freed from the 
two-dimensional text; (2) more interconnect! vity to the text, as a 
result of greater accessibility; (3) closer analysis of the text, as 
the mechanics of textual analysis are simpler; and (4) greater 
fluency in the manipulation of symbols. Many different types of 
assessment can be applied over a period of years. Teachers can use 
"process logs" to track student interaction with literature and the 
new technology. The "blind" reading of student essays can be replaced 
by a consensual assessment model, by which multiple scorers/readers 
assess a given essay simultaneously and anonymously. Among the 
dangers of CMI are that the technology will overshadow the writing 
and that the system will aggravate the inability of many students to 
concentrate. (SG) 
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"'Building the Perfect Beast': Assessing the Effects of CMI 
(Computer-Managed Instruction) on the Teaching of, and 
Student Writing about. Literature" 

Introduction 

The title of my presentation as it appears in the conference 
schedule ("Assessment Methods for Computers in the Teaching 
of Literature") is misleading and unforgivably dull. I deny 
all knowledge of its provenance. Instead, I suggest either 
"Designing a Better Mousetrap" or the title of a mid-1980's 
Don Henley song: "Building the Perfect Beast." The former 
suggests the seriousness of the enterprise as well as its 
possible futility. The latter describes perfectly student 
and faculty attitudes to assessment", a peculiar mixture of 
fear, loathing, and grudging respect for a Godz i 1 la-1 ike 
creature (assessment) running possibly rampant through all 
that has been held dear (teaching techniques of the past). 

The Question 

How are we to assess the impact of computers on student 
writing about literature, where computers are both a 
composing and a teaching tool? 
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The Provisional Answer 

The provisional answer from my perspective: simply do what 
I've been doing in assessment for the last seven years or 
so. Come up with a rating scale; have the students write 
something; and then evaluate their performance. 

The Inadequacy of the Provisional Answer 
Such a strategy sounds easy enough, perhaps; however* it 
won't work because the issue of how to assess the impact of 
computers on student writing about literature is full of 
unknown variafcJes, paradoxes* and problems. It's only a 
slight exaggeration to say that — until we spend a 
significant amount of time thinking through the issue — we 
won't know what we're assessing, we won't know how to assess 
it, and we won't even know why we're assessing it. 

Two Unknown Variables 

1. Heteromedia/hypermedia/mult imedia may (probably will) 
profoundly affect the way students perceive literature. So, 
whatever we're trying to assess will be in a state of flux 
because computer technology shows no sign of voluntarily 
accepting standardization; there's too much profit in 
var iat ion . 
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2. The means of instruction — operating system (OS/2, DOS, 
System 7, or UNIX?), platform (GUI or character- 
based/command line, network or non-network , LAN or WAN?), 
and software (will IBM orphan or upgrade Linkway?) — are 
constantly changing. So, the pedagogy will be in a state of 
flux. One glance at the literature in the field is enough 
to convince anyone of this. 

Four Paradoxes 

1. Until the "massaging" of statistics stage is reached, 
most assessment of writing is done with pencil and paper. 
You can call the pencils "norming wands" and the scoring 
sheets "magic scrolls," but there's still a disturbing 
disjunction between the mechanics of assessment (low tech) 
and the object of assessment: the influence of computers 
(high tech). 

2. Host instructors/teachers know significantly less about 
and are noticeably less comfortable with computers than 
their students are; conversely, students know significantly 
less about and are noticeably less comfortable with 
literature than their teachers are. We have, then, two 
pedagogical gaps rather than the usual one. 

3. The ease of textual production is in inverse proportion 
to the seriousness of the writing error. One might assume 
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that the deftness with which mistakes can be corrected on 
the computer would mean that students would produce cleaner 
copy. On the contrary, the errors frequently aren't fixed, 
so that the written work comes to resemble a palimpsest of 
shifted or mixed constructions. An example from the Nev 
York Times (April 15, 1991) is informative here: 

"The helicopter flew across a runway where a stunt plane had 

just taken taking off." 

The writer had obviously begun here with one construction 
("was just taking off"), replaced it with what he/she 
thought was an improvement ("had just taken off"), but 
forgot to remove all traces of the original version. An 
easy mistake on the computer; an extremely difficult error 
to commit when writing in longhand. 

4. In assessing the effect and/or value of computers in the 
teaching of literature, one is using a hierarchical 
instrument (1-6 norm-referenced scale or a variation of it) 
to assess a fundamentally democratizing* decentralizing* 
decanon iz \ng technology : the computer. Democrat 1 1 iz ing 
because it allows significantly freer and more anonymous 
access to information and communication; decentralizing 
because the modem or LAN/WAN technology allows for distance 
between information and its receiver or producer; 
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decSLnoniz trig because the source of authority (the 
instructor, for example) can very easily remove 
himself /hersel f from the learning process of information 
exchange . 

Four Problems 

1. Literature as written texts is virtually unknown to 
students today. They simply don't read and won't read. 
I've obviously phrased this hyperbol ical ly for rhetorical 
effect, but the crisis in the publishing industry today is 
caused in part by people simply reading less. Any 
instructor knows that every success story (the student who 
reads Uar and Peace and enjoys it) is accompanied by a 
bushel of failures (the student, for instance, who considers 
twenty pages of reading an imposition, or has little or no 
sense of the meaning of what he/she has dust read because 
words have all the friendliness of abstruse mathematical 
calculations, perhaps less). 

2. As assessors, we have to decide whether to be 
proscr iptive, prescriptive, or descriptive. 

3. As assessors of the new technology being used in a new 
way, we have to avoid our proceedings/findings being 
politicized. This is, after all, the Bush era, the era of a 
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President intent on using testing to prove — against the 
evidence — that he is the "Education President." 

4. As users of the new technology, students in their ability 
to access it — their ability, in the crudest form, to use a 
keyboard or to afford their own home computer — will vary 
widely. 

The Question Revisited 

I return then to the question with which I began: How are we 
to assess the impact of computers on student writing about 
literature, where computers are both a composing and a 
teaching tool? 

R evised Answer (in five parts) 

1. We hypothesize beforehand about what the effects of CMI 
will be on the teaching of literature to students: 

A. Greater creativity in approach (because — via 
heteromedia/hypermedia/mult imedia — the text is freed from 
its two-dimensional appearance as symbols on the page). 

B. More interconnect ivity in response (because the 
various parts of the text are more accessible). 

C. Greater willingness to "read" a text closely 
(because the mechanism of textual analysis if simpler than 
with printed texts). 
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D. Greater fluency in the manipulation of symbols 
combined with an increasing reluctance to use the written 
word. 

2. We throw every major form of assessment at students in a 
longitudinal study of several years in duration and see what 
the results yield: 

A. Holistic Assessment (1-6 scale) Lnorm referenced! 

B. Analytic Scoring (isolating particular features) 

Lnorm referenced! 

C. Primary Trait Analysis (focusing on text production 
for particular tasks) Ccr iter ion ba.sedl 

D. Performative Assessment (focusing on text production 
for particular tasks and audiences) Ccr / ter ion bbsedl 

3. We question the students (via surveys) about the precise 
effects of the new technology on their appreciation of 

1 i terature . 

4. We use "Frocess Logs" to track students' interaction with 
literature and the new technology. 

5. We make sure that the technology of assessment matches 
the technology of text production. At its most radical, I'd 
suggest throwing out the old paradigm of "blind" reading 
preceded by extended norming sessions and replacing it with 
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an altogether different paradigm*, the consensual, networking 
mode 1 . 

The Consensual Assessment Model : Here essay scorers/readers 
assess the same essay simultaneously and, via synchronous 
anonymous communication, achieve a consensus on that essay's 
value. In this way, the commun ity of readers and writers in 
which the instructor is marginally primus inter pares is 
matched by a commun ity of assessors in which the bogus 
reconciliation represented by a third reading is replaced by 
genuine agreement achieved after a true meeting of minds. 

H alf-A-Dozen Wor ds of W arni ng 

The new technology will make literature more accessible to 
students by removing its sanctified status as a set of 
revered texts. However, we need to guard against several 
dangers and to assess their rate of occurrence in order to 
judge how seriously under threat written culture may be: 

Danger ttl : That the medium will get in the way of the 
messa$e — the technology can and should be transparent. 

D anger #2 : That the insights about the human condition that 
constitute literature's only claim to privileged status will 
be lost in the process of their multimedia transformation. 
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Danger #3 : That instructors, in their anxiety about 
apparently losing control of the learning process, will pay 
mere lip service to the idea of democratizing the classroom. 

Danger tt4 : That instructors will forget that the goal of any 
language is clear communication — albeit, as in literature, 
clear communication of the ambiguous. 

Danger #5 : That instructors will forget that the value of 
the new technology is as a means to an ends to lead students 
back to the active reading of literary texts. 

Danger tt 6: That the new technology will aggravate the 
greatest weakness students today display, a radical 
inability to concentrate. 

A uthor's Note 

I've begun taking the advice I've presented in this paper in 
a longitudinal study of CMI in the teaching of literature. 
The study is being conducted in collaboration with Locust 
Valley High School, and the results so far augur well for 
the integration of computers into the English curriculum. 
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